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Abstract 

We consider an integro-differential model for evolutionary game theory which describes 
the evolution of a population adopting mixed strategies. Using a reformulation based on the 
first moments of the solution, we prove some analytical properties of the model and global 
estimates. The asymptotic behavior and the stability of solutions in the case of two strategies 
is analyzed in details. Numerical schemes for two and three strategies which are able to capture 
the correct equilibrium states are also proposed together with several numerical examples. 

Key words. Continuous mixed strategies, Replicator dynamics, Evolutionary Game Theory, 
Kinetic equations, Numerical methods. 

1 Introduction 

Evolutionary dynamics is based on the ideas of mathematical game theory In game theory, a 
player's strategy in a game is a complete plan of action at any stage of the game. A pure strategy 
defines a specific move or action that a player will follow in every possible attainable situation in a 
game. A player's strategy set is the set of pure strategies available to that player and defines what 
strategies are available to play. A mixed strategy is an assignment of a probability to each pure 
strategy. This allows for a player to select a pure strategy with a given distribution of probability. 
Since probabilities are continuous, there are infinitely many mixed strategies available to a player, 
even if their strategy set is finite. Of course, one can regard a pure strategy as a degenerate case 
of a mixed strategy, in which that particular pure strategy is selected with probability 1 and every 
other strategy with probability 0. 

In any game, an important concept is the payoff that is the number which represents the 
motivations of a player. The exact definition of the payoff depends on the case of interest: payoff 
may represent profit, utility, or other continuous measures, or may simply rank the desirability of 
outcomes. In all cases, the payoffs must reflect the motivations of the players. Following the basic 
tenet of Darwinism, we may express the success of a player in a game, that means the player's 
survival, as the difference between the player's payoff and the average payoff of all players. 

Dynamic models for continuous strategy spaces have received considerable attention recently 
both in theoretical biology when considering the evolution of species traits [TJ [5j [9] and in economy 
when predicting rational behavior of individuals whose payoffs are given through game interactions 

OT. 

In the present paper we analyze a continuous mixed strategies model for population dynamics 
based on an integro-differential representation. Analogous models based on the replicator equation 
with continuous strategy space were recently investigated in [2j [4j [TUJ [12j [13]. In contrast with 
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finite strategy spaces, where the notion of equilibrium is well understood and studied [HJ[T5j, the 
situation of games with infinite strategies is still missing a general theory due to several technical 
and conceptual difficulties [12J. 

The model here considered is characterized by a continuous density function /(£, q) of popula- 
tion adopting the q G M N strategy at time t and presents some analogies with classical kinetic or 
mean field approaches. In particular we show that the model, which contains a cubic nonlinearity 
in /, can be reformulated in terms of the first moments of the solution. Such reformulation is 
essential in our analysis and in the derivation of numerical approximations. 

For the moment based model we prove global existence of solutions and study the asymptotic 
behavior and stability of solutions in the case of two strategies. Two classes of stationary solutions 
are found. Continuous stationary solutions are characterized by every density function with a given 
mean strategy. If we consider more general solutions, so that the probability distributions are no 
more absolutely continuous with respect to the Lebesgue measure, another class of stationary 
solutions is given by concentrated Dirac masses. Numerical schemes for the two and three strategy 
case which are able to capture the correct equilibrium states are also proposed together with several 
numerical examples. 

The rest of the paper is organized as follows. In Section 2, we present the model for N pure 
strategies and prove a priori estimates and the global existence of solutions. In Section 3, we put 
the emphasis on the model with two pure strategies, which can be reduced to a ID model, and study 
the asymptotic behavior of the solutions and their relation with stationary solutions. Section 4 is 
dedicated to the numerical approximation of the ID model and to numerical tests for the Prisoner's 
Dilemma and for the Hawk or Dove games, with results about the a priori estimate, the asymptotic 
behavior of the solutions and the stationary solutions. In Section 5 and 6, we present the 2D model 
and the numerical tests for the Rock-Scissors-Paper game. Some final considerations are reported 
in the last section. 

2 An integro-differential model for continuous mixed strate- 
gies 

2.1 Setting of the model 

First, we introduce an integro-differential model for continuous mixed strategies. We start from 
some preliminary concepts and definitions taken from [llj. Assume that we have a game where 
there are N pure strategies R\ to Rn and that the players can use mixed strategies: these consists 
in playing the pure strategies R\ to Rn with some probabilities q\ to q^ with qi > and YlQi = 1- 
A strategy corresponds to a point q in the simplex 

N 

Sn := {q = fai, • • • , Qn) G R N : qi > and ^ Qi = 1}. (1) 

i=l 

The corners of the simplex are the standard unit vectors e$ with the i-th component is 1 and all 
others are and correspond to the N pure strategies Ri, i = 1, . . . , N. 

Let us denote by the payoff for a player using the pure strategy Ri against a player using the 
pure strategy Rj. The N x N matrix A = (a^ ) is said to be the payoff matrix. An ^-strategist 
obtains the expected payoff (*4q*)i = J2j a ijQj against a q* -strategist, since <?* is the probability 
that he is met with strategy Rj. The payoff for a q-strategist against a q* -strategist is given by 

N 

A(q,q*) :=q-Af = ^ a^q). (2) 

We consider a population of individuals as a player of the game and denote by /(£, q) the 
density of population adopting the q strategy at time t\ the evolution in time of /, due to the 
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dynamics of the game, is driven by 

dtf(t, q) = f(t, q) Qf A(q, q*)/(i, q*) dcf - «£(/)) , (3) 

where the term 

f A(q,q*)/(t,q*)dq* (4) 

represents the payoff of the strategy q against all the others strategies, A(q, q*) being the inter- 
acting kernel between the q-strategist and the q* -strategist. The last term of the equation ^ is 
defined by 

0(f) ~ [ I /(t,q)^4(q,q*)/(i,q*)rfq*rfq (5) 
and represents the average payoff of the population. 

Since ^2f =1 Qi — 1, we can reduce the number of variables, considering 

N-l 

Qn = 1 - ^ 

i=l 

and obtaining the (N — 1) - dimensional model (J3|, on the simplex 

N-l 

Tn-i := {p = (pi,P2, • • • ,p N -i) e R N ~ 1 \ Pi > , Pi ^ !}» ( 6 ) 

namely 

</>(/) I , (7) 



W, P) = /(t, P) ( / A(p, p*)/(t, P *) dp* 



with A(p,p*) defined by 

iV-l 

^(P, P*) : = P • ^P* = Yl a ijPiPj' ( 8 ) 

and defined by 

#/):=/ / /(t ) p)A(p ) p*)/(t,p*)dp*dp. (9) 

• ' Tn - 1 JTn—i 

Remark 1. // u>e take an initial condition 

/(0,p) = /o(p) >0, (10) 

i /o(p)rfp = 1; ^en is easy to see that f > /or all t > and if fo(p) — /or some p, 
then /(£, p) = for all t > 0. We /mve a/so £/ia£ 



/ /(t,p)dp = l, Vt>0. 

J Tn - i 



(ii) 



T7ws follows from the mass conservation, by integrating the equation ^ w.r.t. p and using J1|) 
and ( flip 

ft/ /(t,p)dp = 0. (12) 
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Let us introduce the moments for /: 

MM) ■= [ P k /(P) dp= [ V \ x P2 2 ■■■ Pn"--; /(p) dp, (13) 

with k := (fei, /u2, . . . , fcjv-i)- Using Mk(/), the payoff and the average payoff ([9| are expressed 
respectively by 

„ iV-l /iV-l \ AT-l 

/ A(p, p*) /(t, p*) dp* = (/) E ^ ^ + ^ + a ^ + E ^ ^ ( 14 ) 

^ r ^-i j=i \i=i / i=i 

N-l /N-l \ iV-l 

0(/)= E Me ,(/) E^ M ^(/) + ^ I + W + E^ M e,(/), (15) 

j=l \i=l / 2=1 

where e$ G IR^ -1 is the standard unit vector with the i-th component equal to 1 and all others 
equal to 0, $ij := a^j — a^jv — a iv,j + ^at,at, <Tj := a/v,j — cln,n, ^% '•= &i,iv — ^at,at. 

In the final form of the equation 0, that we will use later in this paper, the only integral terms 
are the first moments M e . : 

w,p) = /(t,p) ^E^- M ^(/)) ^ + E^ M ej (/)j j . (i6) 



2.2 Global existence of the solutions 



We consider the Cauchy problem (16)-(10) for t > and p G 7jv-i> i- e 



p) = /(*> p) (x>* - M ei (/)) ^ + j2 °u M °> (/)J 



(17) 



i /(0,p) = /o(p), 
with /o(p) > and / Tni / (p)dp = 1. 

Proposition 1 (Local existence). For all M > i/iere earisis T{M) > s«c/i i/iatf i/ 1 |/o(p)| | < M, 
then there exists a unique solution f G C([0,T] x 7jv-i) /or i/ie problem for all T < T(M). 

Proof. Let us define 

T(M) := max min(Ti (R, M),T 2 (R, M) ), (18) 

where 

R 

Tj(i?,M) : = (i? + M ) (1 + ^ + M)(y + Q ( R + M)) . 
T2(i? ' M) : =^M)' 

and 

5(i2, M) := (1 + R + M)(y + 0(R + M)) + (R + M)(V + 6(1 + 2(R + M))) + 2G(R + M) 2 , 



iV-l AT-1AT-1 

V:= J2 N> and 9 := £ £ |^|. (19) 
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We define the set 

B R := {g e C([0, T] x T N -i) \\g~fo\c< R}, 
for R > M, and, for all g G -Br, the operator 



G((/)(p) '=Mp) + J q 9(P) ^(Pi-M e< ((/)) ^+^^Me J (^)JJ dt. 
We have that for all g G I?r, 

|M efc (^)| <it> + M, Vfe, 
|p fc -M efc (^)|<l + i? + M, Vfe. 

It is easy to prove that G(g) G for t < T(M): 

\G(g) - fo\ <f\g-h + fo\ (j2 ^ ~ M ^)\ (\vi\ + E 1^1 \ M *o (^)l)) dt 
<^ (i* + M) ^^(l + fl + M) ^H + (i? + M) E |^,,|^ dt 
= t(R + M)(l + R + M)(V + ®(R + M)) < fl. 



The operator G(#) is a contraction on for £ < T(M): for all g, g G Br 



\G(9)-G(g)\ 



9 I $3 (Pi " ^, (fir)) I Ui + XI M "i (ff) 
i=i V i=i 

JV-1 / AT-1 

s ( X) (p« - Me - (s)) + E M °> (s) 



«/ 

/' 

^0 

/' i» ~ 51 (e |pj - Me - i9)i (j vi + e i^i i M -i(s)iJ j * 

/■ 

./0 



JV-l 



^2\Pi-M ei (g)\ [Wi\ + J2\°u\\ M °M 



JO 



X |p* - ^e t \vi\ + X l<M \M ej (~g)\ 

, i=l V 7=1 

= / t |5-5l (E (l^l+Xl^H M ^(5)ljj ^ 

+ jf Iffl (E MI m -«(») - M ei (ff)|J 

+ jf 1^ I (E (E l Me , (f) - M *;(f)lb* " (^Mf) + w ei (a))|^ 



<tS(R,M) sup\g-~g\. 



The last inequality is obtained using the following inequalities, for all g, g G Br: 

\g\<R + M, 

\M ek (g) - M ek (g)\ < sup\g - g\, Vk 
\M eki (g)M eh2 (g) - M ek2 (g)M ehi (g)\ <2(R + M)sup\g - ~g\ Vfci, k 2 . 



We have that G(g) is a contraction on Br for all t < T(M) and so problem (17) admits a unique 



solution / G C([0,T] x T N -i), for all T < T(M). □ 

We proved the local existence of solution in a time interval (0, T), depending on M. Now we 
define T max as the time limit in which this local solution exists. 

Lemma 2.1. 7/T max < +oo £/ien limsup £ ^ T - ax ||/||<x> = +oc. 

Proof. The proof is by contradiction. Let be limsup^ T - ||/||<x> = M < oo. This means that 
y s > there exists S £ such that Mi G (T max — £ £ ,T max )we have ||/(t)||oo < M + £. Now we fix 



£ > ^ max — minjX, T(M + e)) and consider the problem (17) with initial data (to, /o(p)) = (?,/(£)). 
Using Lemma [l] we have that there exists a solution / G C([0, f + T(M + e)] x 7/v-i) and this is 
in contradiction with the definition of T max because f + T(M + e) > T max . □ 

Lemma 2.2. 77ie solution f of the Cauchy problem (H\ } verifies the following a priori estimate 

||/(t,p)||L~ <max(/o(p))e m , (22) 

P 

Proof. Since < M e< (/) < 1, for all z 

'iV-1 




The proof follows easily using the Gronwall inequality. □ 
Lemma [2TT] and Lemma |Z2| provide the following Theorem: 



Theorem 2.3 (Global existence). There exists a unique global solution f G C([0, +oo) x T/v-i) 
to problem JTT 



Now we present a simple property of the moments that we will use later in the paper to study 
the asymptotic behavior of the solutions for 2 x 2 games. 

Lemma 2.4. If f e C(T N -i) then < M k (/) < I, for all k G R^ -1 . 

Proof. Let S be an open set, such that 

ScTat-i with /(p)>0V P G5. 

The set S is not empty because / > and its integral over T/v-i is equal to 1. We have /(p) > 
P k /(p) > Vp G 5, and so 

M k > / P k /(p)^p>0. (23) 
J s 

We have also that 

0< / P k /(p)rfp< / /(p)dp, (24) 



Since J Tn ^ s /(p) dp > and (24) holds, we have 

1=/ /(P)dp= / /(P)dp+ //(p)dp>M k (/), Vk. 

3 Two strategies games 

Assume there are two different strategies, whose interplay is ruled by the payoff matrix: 



□ 



A 



a ^ 
c d/ " 



In this case the simplex T\ is just the interval [0, 1] and so we have a population where individuals are 
going to play the first strategy with probability p G [0, 1] and the second strategy with probability 
1 — p. The payoff Q is given by 



(25) 



^•p->- 

= (a + d - b - c)pp* + (6 - d)p + (c - d)p* + d 
= app* + /3p + 7P* + (5, 

with 

a := a + d — b — c, /3 := b — d, 7 := c — d, 5 := d. (26) 



The one dimensional Cauchy problem (17) reads 



dtfip) = f(p) [(aM^f) + (3){p - Mi(/))] t>0,pe[0, 1], 
f(0,p) = fo( P ), P€[0,1], 



(27) 



with / (p) > and /„ f {p)dp = 1. 



3.1 Asymptotic behavior of the solutions 

We want to understand what happens asymptotically. We start with a result on the curve of 
change of sign for d t f. 

Proposition 2. If / e C([0, 1]) i/ien /or all t > there exists p = p{t) € (0,1) smc/i that 
Mi (/(*)) =p(t), namely 

dtf(t,p(t)) =0, (28) 
Bga(d t f(t,p)) =-sga(aM 1 (f(t,p))+l3) Vp<p(t), (29) 
sgn(3t/(i,p)) =sgn(aM 1 (/(t,p)) + /3) Vp>p(t). (30) 



The proof of Proposition [2] is easily obtained by Lemma |2.4| 
Let us write the equation for the first moment M\(f): 

M[{t) = (aMi(/) + /3)(M 2 (/) - Af?(/)). (31) 
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The Jensen inequality gets 



Mi(f)=(f\fdp) <j\ 2 fdp = M 2 , 



and so 



8gn(Mj(t))=8gn(aMi (/)+/?)• 
There are four different possible cases: 



(32) 
(33) 



Case a -- 4. (0, 1). Figure 1 
a L_ 

Let us describe in detai. 



shows that it is possible if and only if (a, ft) G A U B. 
the different situations: 





■ 







p=-« 



Figure 1: The quantity 



a 



we have f3 < min(0, — a). The quantity G (0,1) 

a 

—a < f3 < 0; in D we have a < 0, < /3 < —a. 



(0, 1) <==> (a,/3) G A U 5. In A we have /3 > min(0, -a); in 5 



(a, /3) G C U D. In C we have a > 0, 



(a, /3) G A this means that f3 > min(0, -a). If a > then < /3 < aM 1 (f) + /3 < a + /3; if 
a < then < a + /3 < aMi(f) + /3 < /3. In any case we have aMi(f) + /3 > and 
so M{(/) = (aMi(/j + /8)(M 2 (/) - M x 2 (/)) > 0. As shown in Figure J2] (on the left), 
M\{f) is increasing in time and is limited on the right by the curve M(t) — >• 1 with 
M'(t) = (qM + /3)M(1 — M). 

(a, 0) G B this means that j3 < min(0, -a). If a > then /3 < aM r (f) + /3<a + /3<0;if 
a < then a + /3 < aM 1 (f) + /3 < /3 < 0. In any case we have aMijf) + /3 < and 
so M((/) = (aMi(/) + (3)(M 2 (f) - M?(f)) < 0. As shown in Figure [| (on the right), 
Mi(f) is decreasing in time and is limited on the left by the curve M(t) — »• with 
M'(t) = (aM + P)M(l-M). 



P 



Case b e (0, 1). Figure 1 shows that it is possible if and only if (a, (3) GCUD. 
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Figure 2: On the left: evolution of M\(t) in the case (a,/3) G A. On the right: evolution of M\(t) 
in the case (a, /?) G 5. 



(ot, /3) G C Figure [3] (on the left) shows two different situations: 

M 1 (0)>~- =*► M[(f)>0; 
a 

Mi(0)<-^ =► Mf(/)<0. 

By contrast with the previous case, the behavior changes according to the value of the 
first moment Mi at initial time t = 0. If Mi(0) > — £ then M\(t) increases in time 
and is limited on the left by the curve M(t) — > 1 with M'(t) (aM + P)M(1 - M). 
Conversely, if Mi(0) < — £ then Mi(£) decreases in time and is limited on the right by 
the curve M(t) — > with M'(t) (aM + /3)M (1 - M). 



(a, /3) G -D Figure |3] (on the right) shows the two situations: 

Ml (0)>-^ => M 1 '(/)<0; 

Of 

Mi(0)<-^ => Mf(/)>0. 
a 

Also in this case, the behavior depends on the value of Mi(0): if Mi(0) > — £ then 
Mi (t) decreases in time away from the value — £ ; if Mi (0) < — £ then Mi (t) increases 
in time toward the value — -. In any cases, the value —-is the one that dominates in 
time. 




Figure 3: On the left: evolution of M\(t) in the case (a,P) G C. On the right: evolution of M\(t) 
in the case (a, P) G D. 



From the behavior described is easy to understand what happens in the population. If we are in 
region A, there is dominance of the first of the two pure strategies that describe the game, because 
the dynamic encourages the state p = 1 which corresponds to the first pure strategy. This means 
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that in the population, those who adopt the first pure strategy survive, the others do not. In B 
we have the opposite situation: there is the dominance of the second pure strategy and so those 
who adopt the first pure strategy or any other mixed strategy, do not survive. The third region 
C is such that there is not a mixed strategy that dominates, but a priori we can not say which 
of the two pure strategies dominates, it all depends on the value of Mi(0). If Mi(0) > — £ then 
there is the dominance of the first pure strategy, if Mi(0) < — £ then there is the dominance of 
the second pure strategy. In D we have a different situation than in the previous cases: here there 
is coexistence between the two pure strategies and so between the two populations. 



3.2 Stationary solutions 

From the study of the asymptotic behavior, we expect that for t — » oo the solution of the model 



(27) tends to a stationary solution. We can find two classes of stationary solutions: 



Type I If we are in case b, namely — £ G (0,1), then a stationary solution is given by every 
density function f(p) such that 

Mi (/) = -2. (34) 
a 

Type II If we consider more general solutions, so that the probability distributions are no more 
absolutely continuous with respect to the Lebesgue measure, we can say that another class 
of stationary solutions is given by concentrated Dirac masses, i.e.: 

fp(p) = Kv = v)- 

In the following we are going to deal with these generalized solutions in a quite informal way. 
More rigorous arguments will be given in a future paper. 

Here we want just remark that formally, since Mi(fp) = p, we have 

f ¥ (p) [(aMi(/p) + 13) (p - Mi(/p))] = 5(p = p)(ap + f3)(p-p)= 0. 

3.2.1 Linear stability of stationary solutions 

This Subsection is dedicated to the study of the linear stability of stationary solutions. Denote by 

Q(f) = f(p) K«Mi(/) + /3)(p - M 1 (f))j 

the integral operator associated to the replicator equation. Let / be a generalized stationary state. 
We linearise the operator around the state /. So for every perturbation g, with g(p)dp = 0, we 
have the linear operator 



r(f)(g) = lim^o \ (q(/ + hg) - Q(f)] 

= "/Mi(s) + giaM.if) + f3)} (p - M 1 (f)). 



Type I we have — 1 < — < 0. Using (34) we obtain that the linearized equation for a perturbation 

a 

g is given by 

d t g(p) = Q'(J)(g) = 7(p)(p - M,Cf))MM- (35) 

If Jq 1 go(p)dp = 0, the same is true for g for t > 0. 

Proposition 3. Assume — — G (0, 1). Then, there is no continuous stationary solution f(p) 

a 



to problem (27) which is linearly stable. 



10 



Proof. To prove the result it is enough to establish the following equality 

M 2 (/~) = (Mi(/)) 2 , (36) 

which implies that the variance of the measure fdp vanishes and so the measure has to be a 
Dirac mass. We take a continuous stationary solution of Type I, namely a positive function 
/ such that its total mass is equal to 1 and M±(f) = — We perturb this state by a function 
g of zero mass. Computing the first moment of the perturbation g yields 

M{(g) = M 1 (g)(M 2 (f)-(M 1 (f)) 2 ). 

Setting 

Ntif) := M 2 (/) - (M x (7)) 2 = / p( p +P)J(p)dp, 



jo « 

we obtain: 

M 1 (t,g) = M 1 (0,g)e tN K 
Moreover, setting N k (f) := M k+1 (f) - M k (J)Mi(J), we have 

M k (t,g) = M k (0,g) + MlM^l (e ^(/) _ 1} _ 
This means that the condition for linear stability is just: 

Ni(f)= [ 1 p(p+-)7(p)dp<0. 
This inequality can be verified only when the equality condition is satisfied, since we already 



know from (32) that N\(f) > 0. □ 



Type II For the concentrated Dirac masses fp(p) = S(p = p) we have that the linearized equation 
for a perturbation g is given by 

d t g(p) = Q'(fp)(g) = 9(p)(<*p + P)(p-p). (37) 

Proposition 4. The Dirac mass solutions are linear stable if, on the support of g(p), we 
have: 

{ap + p)(p-p) <0, Vpe [0,1]. 
For general perturbations, i.e.: with supp g = [0, 1] we have three cases: 

1. the Dirac mass concentrated in p = is stable if j3 < 0, which means in the original 
constants, b < d. 

2. the Dirac mass concentrated inp—\ is stable if a + j3 > 0, which means in the original 
constants, a > c. 

3. if — 1 < — < 0, then the Dirac mass concentrated inp=—- is stable. 

4 Numerical approximation of the ID model 

We want to perform some numerical simulations with model p7| ). We consider some nodes 

Pi e [0,1] z = 0,...,/, (po = 0,p/ = l), 

and, using the trapezoidal rule, we obtain the following quadrature formula for the first moment 
Mi(/) (for all times t e [0, T]) 

Mi(f)= / pf(t,p)dp « Mi(/) := Y. Pl+ \ ~ \Pi+iffoPi+i) +Pif&Pi)] • (38) 
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Clearly the above discretization is such that conservation of mass holds 

j^p_^_Pi [fitjPi+i) + f{tjPt)] = 

i=0 

1-1 

(aMi(/) + P) J2 Pi+ \~ Pi \{p i+ i - Mi(f))f(t,p i+1 ) + fa - Mi (/))/(*, = 0, 
i=o z 

provided that initially 

gmi^i [/ofe+i) + /ofe)] = L 
i=o z 

In the case of equally spaced points, Ap = p i+ i — p iy the above property implies that 

f(t,Pi) < ^ Vt >0, z = 0,...,I, 

and thus the numerical solution is well-defined even when we approach a Dirac delta at the con- 
tinuous level. More precisely both possible steady states are preserved by the numerical method, 
namely M\{f) = —f3/a and the discrete Dirac delta defined as 

0, i+3 

fpi(Pj) = \ A P 
2 

— , i = j, i = 0,1. 

For the time discretization we simply use a fourth order Runge-Kutta method with constant time 
stepping At on the interval [0, T]. Nonnegativity of the numerical solution is achieved taking 

At^laMiCD+^l" 1 , 

where f n ( Pi ) = f{t n , Pi ), t n = nAt. 

4.1 Numerical tests: Prisoner's Dilemma game 

One interesting example of a game is given by the so-called Prisoner's dilemma game in which there 
are two players and two possible strategies. The players have two options, cooperate or defect. 
The payoff matrix is the following 



A 



c 


D 


R 


S 


T 


P 



c 

D 



If both players cooperate both obtain R fitness units (reward payoff); if both defect, each receives 
P (punishment payoff); if one player cooperates and the other defects, the cooperator gets S 
(sucker's payoff) while the defector gets T (temptation payoff). The payoff values are ranked 
T > R > P > S and 2R > T + S. From the game theory we know that cooperators are always 
dominated by defectors. One of the main problems has been about the possibility of success for 
cooperation, which is impossible in the pure strategies models: the replicator dynamics of prisoner's 
dilemma, [TT1 . shows that cooperators are extinguished. 

For the numerical tests we fix the following normalized payoff matrix: 

*=(l ?V ( 39 ) 



b e / 

with b = 1.1 and e = 0.001. In this case we have a = 1 — b + s < and B = — e < and so - > 0. 

' a 

This means that stationary solutions are expected to be given by concentrated Dirac masses (see 



Section 3.2). For general perturbation we have that p = is linearly stable. 



12 



4.1.1 Test n.l 

We consider the initial datum 



/o(p) = l Vpe[0,l]. (40) 
Figure [4] shows that the density / tends to concentrate at the point p = 0, according to what we 



Evolution of the density f for the Prisoner's Dilemma Game 




Figure 4: Prisoner's Dilemma Game, test n.l: b = 1.1, e = 0.001. Plot of the evolution over time 
of for the Cauchy problem ^\ with f (p) = 1 for T = 1000. 



expected. 



We have studied, numerically, the L°°-norm of the solution /. Using the a priori estimate (22), we 
know that 

<||/o(p)||ooe(l"l+IW* = e (N+IW*, 



and this yields 



H(t) := Log(\\f\\oo) < (M + |/3|)e Lo ^ =: E(t), Vt. 



(41) 



Figure [5] shows that inequality ( [41] ) is respected. In the prisoner's dilemma game we have that 



Evolution of the L infinity norm of the density f 




Figure 5: Prisoner's Dilemma Game, test n.l: we have Log(t) on the x-axes and H(t) and E(t) on 



the y-axes for the Cauchy problem (27) with fo(p) = 1 for T = 1000. 
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The curve M1 (t) fo the Prisoner's Dilemma Game 




0.3 0.35 
M1 



Figure 6: Prisoner's Dilemma Game, test n.l: b = 1.1, e = 0.001. Plot of the evolution of M\(t) 
vs. time t for the Cauchy problem (27) with fo(p) = 1 for T = 1500. 



(a, /?) G 5 (see Figure [I]) and we know, from game theory, that the defectors' pure strategy dom- 
inates the cooperators' pure strategy. The evolution in time of M\(f) (Figure |6| is as expected 
(see Figure [2] (on the right)). 



We consider now a quadratic initial datum for the model (27). We have plotted the numerical 
results in Figure [7] As in the previous case with fo(p) = 1, we see that the density / tends to 
concentrated at the point p = that corresponds to the defectors' strategy. 



Evolution of the density f for the Prisoner's Dilemma Game 




■ initial 
t=250 

■ t=500 
- final 



Figure 7: Prisoner's Dilemma Game, test n.l: b = 1.1, e = 0.001. Plot of the evolution over 



time of f(t,p) for the Cauchy problem related to the model (27) with initial datum fo(p) 



.2,2 



p + 1 y p G [0, 1] for T = 1000. 



4.1.2 Test n.2 

Now we want to consider an initial datum fo(p) with compact support in [pi,P2] C [0,1], with 
Pi < P2 • If we define 

9 :=^- (42) 

P2 - Pi 
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we have that fo(q) has compact support in [0, 1]. W.r.t. q the average payoff (25) has the following 
form 

A(q,q*) = aqq* + Pq + tyq* + 5, 

with 

a := a(p 2 - Pi) 2 j3 := a(p 2 -pi)pi+P(p2 ~Pi), 
7 : = a(p2 - Pi)pi + l(p2 ~ Pi), 6 := ap\ + (7 - /3 - 2£)pi + £. 

The quantity 

1/ , /T 



(43) 

(44) 
(45) 



Pi + - 

« P2-P1 \ OL 



is positive if £ > 0, as in the Prisoner's Dilemma game. This means that the point p — p\ 
(corresponding to the point q — 0) is stable, as shown in Figure [8] that is related to the Cauchy 
problem (|27|) with the following initial datum: 



fo( P ) 



2 Pe[|,|]u[i,i] 

elsewhere. 



(46) 



Evolution of the density f for the Prisoner's Dilemma Game 




Figure 8: Prisoner's Dilemma Game, test n.2: b = 1.1, e = 0.001. Plot of the evolution over time 



of f(t,p) for the Cauchy problem (27) with initial datum (46) for T = 1000. 



4.2 Numerical tests: Hawk or Dove Game 

Another example of a game is given by the so-called Hawk or Dove game in which there are two 
pure strategies: hawks (H) and doves (D). While hawks escalate fights, doves retreat when the 
opponent escalates. The benefit of winning the fight is b. The cost of injury is c. If two hawks 
meet, then the expected payoff for each of them is The fight will escalate. One hawk wins, 
while the other is injured. Since both hawks are equally strong, the probability of winning or losing 
is \. If a hawk meets a dove, the hawk wins and receives payoff 6, while the dove retreats and 
receives payoff 0. If two doves meet, there will be no injury. One of them eventually wins. The 
expected payoff is |. Thus the payoff matrix is given by 

H D 
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If b < c, then neither pure strategy is a Nash equilibrium. If everybody adopts the first pure 
strategy (H), it is best to adopt the second pure strategy (D) and vice versa. This means that 
hawks and doves can coexist. Selection dynamics will lead to a mixed population. 

We fix b = 1 and look for a suitable value of c > b for the numerical tests. We obtain the 
matrix 

/ l Z± i \ 

2 1 



A 



V 







2 / 



In this case we have 



1 

a 



1 



and (a, f3) G D if c > 1 (see 



Figure . 



The function 



fo(p) = -p 2 + P +l 



(47) 



is an admissible stationary solution for the problem, namely positive, with a total mass equal to 1, 
and with the first momentum equal to — — , if and only if 9 = | and c = || . In Figure 



we show 



the numerical results for the Cauchy problem (27), associated to this initial datum. We remark 



that the numerical scheme preserves the stationary solution. 



Evolution of the density f for the Hawk or Dove Game 




Figure 9: Hawk or Dove Game, quadratic stationary solution: we plot the evolution over time of 



f(t,p) for the Cauchy problem p7k with initial datum (47) for T = 1000. 



As a consequence of Proposition [3j the stationary solution (47) is not linearly stable, in fact 



jf((- 



M 2 (/ (p)) - (M 1 (f (p))Y = I [[ ' + -/r +P)[P-^.)) dp = 0.07. 
Actually, even small perturbations of the datum can generate large perturbations on the solutions. 



We consider a perturbation with zero mass for the function (47): 

o 2 



/o(p) 



~P 



-p+ 1 + 0.02sin(27rp), 



(48) 



V p G [0, 1]. Figure [To] shows the evolution of f(t,p) for the related Cauchy problem. The perturbed 
datum ([48]) originates the loss of the stationary solution, as seen in Figure 10 The solution of 



the problem evolves (slowly) towards a Dirac mass and we can see the first moment Mi which 
converges to the value — - = 0.4722 (Figure 11), as expected since (a, /?) are in the region D. 
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The curve M1 (t) fo the Prisoner's Dilemma Game 
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Figure 11: Hawk or Dove Game, asymptotic behavior: we have t on the y-axes and M\{t) on the 



x-axes for the Cauchy problem (27) with initial datum (48) for T = 1000. 
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5 Three strategies games 

Assume there are three different strategies, whose interplay is ruled by the payoff matrix: 

(a\ a 2 a 3 \ 
a 4 a 5 a 6 . 
a 7 a 8 a 9 J 

We have a population where individuals are going to play strategy A with probability pi, 
strategy B with probability p 2 and strategy C with probability 1 — p\ — p 2l for (^1,^2) £ where 
the simplex T 2 is just 



T 2 = {p = (pup 2 ) e M 2 \p u P2 > 0, pi +p 2 < 1}. 
The payoff ([2| is given by 



(49) 



^4(P,P*) 



Pi 

P2 





a 2 




)G 


a 5 


3( 




a 8 





Pi 

P2 



= (ai - a 3 - a 7 + + («2 - a 3 - a 8 + a 9 )pipl 

+(a 4 - a 6 - a 7 + a 9 )p\p 2 + (a 5 - a 6 - a 8 + a 9 )p 2 p 2 + («3 - 09)1*1 

+(a 7 - a 9 )p* + (a 6 - a 9 )p 2 + (as - «9)p$ + a 9 

= ap x p\ + + 7P1P2 + fepj + <*Pi + VPi + CP2 + M + ^ 



(50) 



with a := ai — a 3 — a 7 + ag, /3 := a 2 — a 3 — a 8 + ag, 7 := a 4 — a$ — a 7 + ag, (5 := a 5 — ae — a 8 + ag, 
a := a 3 — ag, 77 := a 7 — ag, £ := a§ — ag, /i := a 8 — ag and £ := ag. In this case, the problem (17) is 



\d t f(p) = F(f) t>o lP er 2 , 
\/(0,p) = /o(p), 
where the source term F(f) is defined as follows: 

F(f) := f(p)[(aM (lt0) (f)+/3M (0tl) (f) + a)( Pl -M {m (f)) 
+ (7M ( i,o)(/)+«M (0il) (/)+0(P2 -M (0il) (/))]. 

We consider the initial datum /o(p) such that /o(p) > and J!^ /o(p)dp = 1. 

Remark 2. is easy to prove that if 



R >0 

a — 7 p 

then every distribution function f(p) with 



M(i 



(1,0) 



S a — 7/3 



and 



and 



(77 — 

5 a — 7/3 



>0, 



(0,1) 



(77 — 
(5a — 7/3 



(51) 



(52) 



(53) 



is a stationary solution for the problem (51). Actually, by arguing as in Section 3, also Dirac 
masses concentrated on points are stationary solutions of these equations. 
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5.1 A special case: the Rock-Scissors-Paper Game 

We consider the Rock- Scissors-Paper game, which is characterized by having three pure strategies 
such that R\ is beaten by i?2, which is beaten by i?3, which is beaten by R\. The outcomes of the 
game are tabulated as 

1 -1> 

A= ( -1 1 | . (54) 

1 -1 



In the Rock- Scissors-Paper game, the constants that appear in the source term (52) have the 
following values: a = 0, f3 = 3, a = — 1, 7 = —3, £ = 0, £ = 1. and so 

fit; — a 5 aj — ^a 1 
5 a — j (3 5 a — 7 j3 3 

The initial datum /o(p) = 2 has integral over T2 equal to 1 and the moments Mn )(2) 
1 



^(0,1) (2) = - and so it is a stationary solution for this game. In the next Section 
o 

present the numerical results related to this stationary solution. 



we will 



Now we want to present a result about the curve of changing sign for dtf: for this game the 
source term (l52l) is 



F(f) = f [(3M (0il) (/) - l) Pl + (1 - 3M (1)0) (/))p 2 + M (1)0) (/) - M (0)1) (/)], 
and so we have that the curve p(t) C M? such that d t f(p(t)) = 0, is given by 

Pi(3M (0l i) -1)+P2(l-3M (1>0 ))+M ( i i0 ) -M (0 ,i) =0, V(M (1)0) ,M (0)1) ) eT 2 , (55) 

that is the straight line joining the points (M(i ?0 )(/), M(q,i)(/)) and -'- n ^ ne following 

Section [6] we will present the evolution over time of this straight line. 



6 Numerical approximation for the 3-strategies model 



First we want to construct a numerical method for problem ( |51[ ). The domain T2 is just the 
triangle with vertices (0,0), (1,0), (0,1). In order to make the numerical integrations, we fix a 



discretization step Ap and a uniform triangular grid in T2 as Figure 12 shows. Each point of the 
grid is 

Pij := (pi,i,P2,j) z = 0, . . . I, j = 0, . . . ,I-i, (56) 
with / := . We use the notation gij := #(t,Pi,i,P2,j) for alH = 0, . . . , / and j = 0, — 2 

to indicate the value of a general function g(t,pi,p2) at each grid point p^-. In order to discretize 
the integral of g over the domain T2, we start to consider each triangle of the grid and indicate its 
vertices as (x s ,y 3 ), for s = 1,2,3. We define the following quantities: 

g := max(g(x s ,y s )) 5 = 1,2,3 (57) 
g:=mm(g(x 8 ,y a )) 5 = 1,2,3, (58) 

the maximum and the minimum value of g on the triangle. On each triangle of the grid we consider 
a 2D product formula based on the trapezoidal rule: 

pi rl—pi 

g(t,p u p 2 )dp= / g(t,p 1 ,p 2 )dp 2 dpi 



r 2 



Ap 2 
2 



I-l I-i 

EE 

2=0 j=0 



1-1 i-i 



&i-Si)J+2J2J 

1 j=0 



5 9 + n(92 ~9n) 



(59) 
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Triangular grid for the domain T2 




where 



Figure 12: An example of triangular grid for the domain 7^- 

g x = max(^ ;J ,^ )j+ i,^ +lj ), g 1 min(^ ?j , g iij+ i, gi+ij), 
g 2 = max(^j,^ )i+ i,^_i )i+ i), g 2 = mm(gij,g iij+1 ,gi- lij+1 ). 



(60) 
(61) 



The discretization of the first moments M( 1?0 ) and M( ,i) is easily obtained by (59), considering 
the function g(t,p 1 ,p 2 ) = pif(t,p u p 2 ) for M ( i j0 )(/) and g(t,p 1 ,p 2 ) = p 2 f(t,pi,pZ)foi M (0j i)(/). 

Similarly to the one-dimensional case it can be shown that the method preserves the total mass 
in time, as well as discrete analogous of the steady states. As before the time discretization is done 
with a fourth order Runge-Kutta method with constant time stepping. 

6.1 Numerical tests: The Rock-Scissors-Paper Game 

We consider the Cauchy problem associated to the problem ([51]) for the Rock- Scissors-Paper game 



with the payoff matrix (54). The problem has the following equation: 



dtf(p) = /(P)[(3M (0)1) (/) - l)(p! - M (1 ,o)(/)) 



+(1-3M (1 , )(/))(P2- 
L/(0,p) = /o(p)- 



M (0 ,i)(/))], 



(62) 



6.1.1 Test n.l 



We start with an initial datum compactly supported in 7-2: the sum of s > 1 truncated Gaussian 
functions, centered in (Pi' r ,P2' V ) for r = 1, . . . , s. To ensure that (11) applies, we normalize the 
datum so that its integral over 7i is equal to 1. Our datum is of the following type 



/o(p) := 



G(p) 



Jr 2 G(p)dp' 



where 



with K r > 0. 



S ( 1 

G(p) :=^maxf — , 

r=l ^ 



-K r [(Pi-P 1 n 2 +(P2-p° 2 - r ) 2 ] 



0.01,0 



(63) 



(64) 
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The initial datum for the density f 



The density f at time t 
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Figure 13: Rock-Scissors-Paper Game, test 1.1: the evolution of the density / that is the numerical 



solution of the Cauchy problem (62)-(63) with 8 = 2, (Pi' 1 ,^' 1 ) = (j^, §), (p?' 2 ,^' 2 ) = (§, 
Ki = 300, K 2 = 190 and T = 20. 



Test 1.1 

We fix s = 2, (p°{\p°/) = (^, |), (p?' 2 ,P2' 2 ) = (f , To), *i = 300, K 2 = 190. 



The graphical results (in Figure [13]) shows that there is dominance of one of the groups: the 
initial datum is likely to have two areas of concentration, the final configuration shows only one 
area of concentration, and the total L 1 mass remains constantly equal to 1 over time. The dom- 
inance group is contained in the region where d t f is positive as we can see in the Figure [T4| that 
shows the contours of / and the numerical results of the straight line p(t) of changing sign for dtf 



(see Subsection 5.1). We also remark that, for all £ > 0, the support of f(t) is equal or a subset of 



the support of the initial datum f : 

supp(f(t)) C supp(fo), W > 0. 



Test 1.2 

We fix s = 2, (p°{\p°/) = (§, |), (p°{\p /) = ( K x = 600, K 2 = 300. 

The graphical results (Figure [l5] and Figure [l6| show that the situation is different from the 
previous test: in this case the initial datum lies between the two regions where d t f is positive 
and negative and the straight line p(t) of separation between this two regions does not changes 
significantly over time. Therefore the configuration of the function / at the final time T is not 
very different from that at the initial time. 



6.1.2 Test 1.3 

We fix a = 3, (p°{\p°/) = (I |), (P {\p /) = (P i 3 ,p /) = (± |), K x = 600, K 2 = 300 

and K 3 = 300. 
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Figure 14: Rock-Scissors-Paper Game, test 1.1: the evolution over time of the contours of the 
density / and of the curve p(t) of change of sign for d t f for the Cauchy problem (62)- (63) with 
the same data and the same parameters as the previous Figure [13] 



The graphical results (Figure 17 and Figure 18) show dominance phenomena in the region where 
dtf is positive. Initially the three areas of concentration are located, almost entirely, in the region 
where d t f is negative. The time evolution shows us that, already at t = 6, two of the three areas of 
concentration are in the middle between the two regions where dtf is negative and positive, then 
to the final time, are completely in the region where d t f is positive. So it is clear that dominance 
takes place in these areas. 



7 Conclusions 

We have considered a kinetic-like model for the evolution of a continuous mixed strategy game. 
The model is based on the time evolution of a density function describing the density of population 
adopting a given strategy. We established several analytical properties and develop some numerical 
discretizations useful for numerical simulations in the case of two and three strategies. Several 
explicit examples for two and three strategies games are reported. Of course when considering 
more strategies a deterministic approach may result in excessive computational requirements and 
stochastic simulations methods should be considered [7]. 

Let us finally mention that, in the situation considered so far, each player adopts a strategy 
and evolution over time leading to survival or not of the player. In principle it can be interesting 
to consider a situation in which each player can change strategy by a random mutation, so moving 
through the strategy space. One can introduce, to this end, a term in the equation that allows 
for the random change of strategy, following the ideas presented in |8J. The most natural way to 
model this phenomenon is to add a variation term in the equation ( [16] ) ? due to the probability 
P G Tn-i 

d t f-DA p f = f ^(Pi-Me,(/)) ^+^^M ej .(/)j j , (65) 

with D > 0. The new term A p / can be interpreted as a diffusion term describing the spreading 
of the population in the probability space from strategy to strategy, which in evolution models 
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The initial datum for the density f 
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Figure 15: Rock-Scissors-Paper Game, test 1.2: the evolution of the density / that is the numerical 
solution of the Cauchy problem ( 
K x = 600, K 2 = 300 and T = 20. 



solution of the Cauchy problem (|62|)-(|63|) with 8 = 2, (Pi' 1 ,^' 1 ) = (|, |), (p?' 2 ,^' 2 ) — 



corresponds to a random mutation mechanism, and will be the object of a future work. A similar 
model has been presented recently in [14J. 
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Figure 17: Rock-Scissors-Paper Game, test 1.3: the evolution of the density / that is the numerical 



solution of the Cauchy problem (|62|)-(|63|) with s = 3, (p?' 1 ,^' 1 ) = (|, |), Cp?' 2 ?^' 2 ) = (lip 



(P?' 3 ,P2' 3 ) = (lo> f )> = 600 > ^2 = 300, if 3 = 300 and T = 20. 




Figure 18: Rock- Scissors-Paper Game, test 1.3: the evolution over time of the contours of the 
density / and of the curve p(t) of change of sign for d t f for the Cauchy problem (62)- (63) with 
the same data and the same parameters as the previous Figure [17[ 
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